Thermal measurements of the face can be utilized for various applications, which include detection of various physiological responses and/or medical conditions. However, when taken in uncontrolled real-life scenarios, thermal measurements can be influenced by various intrinsic and extrinsic factors. Some examples of such factors include infrared radiation directed at the face (e.g., sunlight), wind, physical contact with the face (e.g., touching the face with the hand), and consumption of substances such as various medications, alcohol, caffeine, and nicotine. Though these factors may be unrelated to a physiological response and/or medical condition being detected, they may nonetheless affect the temperatures at various regions of interest on the face. Therefore, these factors may be considered confounding factors that hinder the detection of the physiological response and/or medical condition. Thus, there is a need for systems that utilize thermal measurements collected in real-life scenarios to account for various intrinsic and/or extrinsic factors that may alter the thermal measurements, in order to avoid the detrimental effect that such factors may have on the utility of the thermal measurements for various applications.